Learning From Stories: Using Crowdsourced Narratives to Train Virtual Agents

In this work we introduce Quixote, a system that makes programming virtual agents more accessible to nonprogrammers by enabling these agents to be trained using the sociocultural knowledge present in stories. Quixote uses a corpus of exemplar stories to automatically engineer a reward function that is used to train virtual agents to exhibit desired behaviors using reinforcement learning. We show the effectiveness of our system with a case study conducted in a virtual environment called Robbery World that simulates a bank robbery scenario. In this case study, we use a corpus of stories crowdsourced from Amazon Mechanical Turk to guide learning. We evaluate Quixote under a variety of different conditions to determine the overall effectiveness of the system in Robbery World.